Better average - case predictions fork - binary

Papers by Goldman, Rivest and Schapire, and Goldman and War-muth contain worst case analyses for several polynomial time algorithms for predicting k-binary relations. Here we present an empirical study of the average-case performance of those algorithms, and also of two new methods which dominate them in this average-case setting. The better of the two methods uses a graph coloring heuristic to solve for the target matrix row types. 1 The k-binary relation prediction problem In the k-brpp, an adversary chooses a hidden nm matrix of bits. Only k rows of this matrix are unique; the remaining n ? k rows are copies of these rows. Then, n m times, the adversary selects one of the covered bits in the matrix and asks us to predict its value. After guessing, we get to see the actual bit. One performance measure is the total number of mistakes that we make during the n m predictions for a given matrix. A more detailed measure is, for every t, the average accuracy of the prediction which occurs after t bits have been uncovered. In the average case experiments which are the subject of this paper, the adversary is a random process, i.e. the target matrices and the query orders are randomly generated. 1 2 Average case performance of earlier algorithms Suppose that part of the target matrix has been uncovered, and we have been asked to predict the bit which lies at matrix location (i; j). The only way to make a correct prediction is to predict the value at a location (i 0 ; j), where i 0 is a row which is of the same type as row i but which, unlike row i, is deened in column j (that is, where the entry in column j has already been uncovered). Unfortunately, we can rarely be sure that two rows of the partially uncovered matrix are of the same type. All of the diierent algorithms discussed in this paper amount to diierent methods for dealing with this uncertainty. Broadly 1 Speciically, we drew from a uniform distribution over all matrices with exactly n=k copies of each row type.